Data set Replication Code for "Local Standards, Behavioral Adjustments, and Welfare: Evaluating California’s Ocean-Going Vessel Fuel Rule"
Richard Klotz and Julia Berazneva
Updated 9/2021

Requirements
-	Steps 1, 5i, 5ii, 5iv, 5v and 5vii use python 2.7 and the ArcGIS arcpy package 
-	Remaining python scripts use python >= 3.5 
- 	Paths to appropriate files need to be updated in each file discussed below 

Two pieces of our dataset are obtained from commercial vendors. Information on our sources are below, but these are easily obtained from a variety of sources.
1) Vessel characteristics database - We obtained vessel characteristics from Clarksons Research (World Fleet Registries, current and various historical reports) and supplemented these records with data from Marine Traffic (https://www.marinetraffic.com/). Required data fields are: IMO number, vessel classification, design speed, fuel consumption at design speed, main engine type, main engine power, deadweight, length, width, draught, beam, year built, flag country and operator. Auxilliary engine information is discussed in paper
2) Maritime fuel prices - We obtained daily Ex-Wharf prices in Los Angeles for Marine Gasoil (0.1%) and IFO 380 (3.5%) from S+P Platts. The fuel price data is merged in after the final voyage data set is created.

The steps below explain how to reconstruct the final datasets. Many of these steps --- particularly those dealing with the AIS data in 5) --- are time intensive (e.g., days of processing). We have provided all AIS inputs to step 6) -- with the exception of vessel characteristics -- to construct the final datasets. These are in folders that end with "_v2a". Some of these files will have to be unzipped and placed in the appropriate directories (or the file paths in step 6) will have to be modified).

STEPS: 
1) Run “writePortBuffers.py” to create port buffers and study area boundary layer

2) Obtain and clean vessel characteristics database

3) Construct marginal damages grids (provided in data/damage_grids)

4) Clean USACE entrance/clearance data using "cleanEC.do"

5) Obtain AIS data and construct vessel voyages (some of these steps are time intensive)
	i) Download AIS data from Marine Cadastre (https://marinecadastre.gov/ais/) and process points into trajectories
		Download is automated using “download_AIS.bat”, which requires wget
		For 2009-2014: use “createTracks_month.py” for 2009 to 2014 this will loop through by month and apply track creation algorithm 
		For 2015 and later: use "cleanAIS_1516.py" to unzip an create necessary tables; then run “runTrackBuilder_1516.bat”
		Parameters in “AIS_TrackBuilder_v3.1_64bit_modified.py” and “TrackBuilderFunction_M.py” must be consistent
		
	ii) Use “merge_tracks_AK_HI.py” to clean trajectories
		This joins trajectories that are split by month
		And interpolates trajectories between AK/HI
		Interpolated tracks are added as new tracks, original tracks are maintained but flagged

	iii) Use "get_interp_times.py" to create list of times voyages are interpolated so that these points can be flagged in later steps
		This step requires python >=3.5 because it uses geopandas
	
	iv) Use “process_lines_to_voyages.py” to process full trajectories in to routes (origin-destination pairs)
		This script breaks trajectories apart based on port buffers and study area boundaries
		Creates two sets of routes, those broken at study area boundary and those not (in "*no_bound" files)
	
	v) Use "add_fuel_damages_to_voyages.py" to conduct speed, fuel, and marginal damage calculations
		This script:
		a) Down samples points along tracks - calculates fuel calculation for each segment
		b) Down samples further - determines marginal damage at each downsampled point
		c) Aggregates to track level and saves as csv
		d) Saves downsampled points as csv
		e) Uses downsampled data to create speed bins and saves as csv

	vi) Use "add_fuel_damages_to_voyages_no_bound.py" to conduct similar calculations on entrances/exit tracks using the no_bounds files 

	vii) Recover MMSI-IMO mapping for 2010-2014 (b/c scrambled in raw AIS data)
		a.	load all observed MMSI-IMO pairs in AIS data (“create_vessel_list.py”)
		b.	run "run_mmsi_imo_matching.do" to load all voyage data and conduct matching to recover IMO numbers 
		
6) Use “create_final_data.py” to merge voyage data (points, speed bins, voyage totals) with MMSI-IMO mapping, vessel characteristics, entrance/clearance data and creates final voyage and speed bins data sets 
	This creates a number of different output datasets
	i) main voyage level dataset
	ii) speed bins dataset
	iii) within port dataset
	iv) raw points data
	v) points data with grid points assigned for heat maps
	
7) Run "add_chars_to_voyages.py" to add vessel characteristics and routes to spatial voyage data
	This saves a single shape file of voyages with route, vessel and ECA information
	This file is then used for mapping
	
